[SPARK-7763] [SPARK-7616] [SQL] Persists partition columns into metastore by liancheng · Pull Request #6285 · apache/spark

liancheng · 2015-05-20T17:15:17Z

No description provided.

AmplabJenkins · 2015-05-20T17:17:11Z

Merged build triggered.

AmplabJenkins · 2015-05-20T17:17:18Z

Merged build started.

SparkQA · 2015-05-20T17:19:14Z

Test build #33148 has started for PR 6285 at commit 2a38b5b.

SparkQA · 2015-05-20T19:11:14Z

Test build #33148 has finished for PR 6285 at commit 2a38b5b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-20T19:11:18Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-20T19:11:19Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33148/
Test PASSed.

AmplabJenkins · 2015-05-20T23:17:11Z

Merged build triggered.

AmplabJenkins · 2015-05-20T23:17:21Z

Merged build started.

SparkQA · 2015-05-20T23:17:44Z

Test build #33184 has started for PR 6285 at commit 7e02910.

SparkQA · 2015-05-21T00:14:52Z

Test build #33184 has finished for PR 6285 at commit 7e02910.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T00:14:56Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-21T00:14:56Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33184/
Test FAILed.

AmplabJenkins · 2015-05-21T04:22:10Z

Merged build triggered.

AmplabJenkins · 2015-05-21T04:22:19Z

Merged build started.

SparkQA · 2015-05-21T04:24:19Z

Test build #33217 has started for PR 6285 at commit 387153c.

…se/SPARK-7616.

AmplabJenkins · 2015-05-21T04:47:12Z

Merged build triggered.

AmplabJenkins · 2015-05-21T04:47:19Z

Merged build started.

SparkQA · 2015-05-21T04:48:05Z

Test build #33219 has started for PR 6285 at commit 6733276.

SparkQA · 2015-05-21T04:49:44Z

Test build #33219 has finished for PR 6285 at commit 6733276.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T04:49:45Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-21T04:49:46Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33219/
Test FAILed.

SparkQA · 2015-05-21T05:23:37Z

Test build #33217 has finished for PR 6285 at commit 387153c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T05:23:41Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-21T14:00:48Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33247/
Test FAILed.

AmplabJenkins · 2015-05-21T14:22:11Z

Merged build triggered.

AmplabJenkins · 2015-05-21T14:22:21Z

Merged build started.

SparkQA · 2015-05-21T14:24:20Z

Test build #33248 has started for PR 6285 at commit d677f7d.

SparkQA · 2015-05-21T16:19:57Z

Test build #33248 has finished for PR 6285 at commit d677f7d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T16:20:01Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-21T16:20:02Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33248/
Test PASSed.

AmplabJenkins · 2015-05-21T17:42:12Z

Merged build triggered.

AmplabJenkins · 2015-05-21T17:42:22Z

Merged build started.

SparkQA · 2015-05-21T17:43:07Z

Test build #33257 has started for PR 6285 at commit bb2829d.

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with #6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes #6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with #6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes #6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory (cherry picked from commit 8730fbb) Signed-off-by: Yin Huai <yhuai@databricks.com>

SparkQA · 2015-05-21T18:15:17Z

Test build #846 has started for PR 6285 at commit bb2829d.

SparkQA · 2015-05-21T19:37:18Z

Test build #33257 has finished for PR 6285 at commit bb2829d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T19:37:22Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-21T19:37:22Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33257/
Test PASSed.

SparkQA · 2015-05-21T20:18:40Z

Test build #846 has finished for PR 6285 at commit bb2829d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yhuai · 2015-05-21T20:49:50Z

I am going to merge it to master and branch 1.4.

…tore Author: Yin Huai <yhuai@databricks.com> Author: Cheng Lian <lian@databricks.com> Closes #6285 from liancheng/spark-7763 and squashes the following commits: bb2829d [Yin Huai] Fix hashCode. d677f7d [Cheng Lian] Fixes Scala style issue 44b283f [Cheng Lian] Adds test case for SPARK-7616 6733276 [Yin Huai] Fix a bug that potentially causes https://issues.apache.org/jira/browse/SPARK-7616. 6cabf3c [Yin Huai] Update unit test. 7e02910 [Yin Huai] Use metastore partition columns and do not hijack maybePartitionSpec. e9a03ec [Cheng Lian] Persists partition columns into metastore (cherry picked from commit 30f3f55) Signed-off-by: Yin Huai <yhuai@databricks.com>

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with apache#6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes apache#6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory

…tore Author: Yin Huai <yhuai@databricks.com> Author: Cheng Lian <lian@databricks.com> Closes apache#6285 from liancheng/spark-7763 and squashes the following commits: bb2829d [Yin Huai] Fix hashCode. d677f7d [Cheng Lian] Fixes Scala style issue 44b283f [Cheng Lian] Adds test case for SPARK-7616 6733276 [Yin Huai] Fix a bug that potentially causes https://issues.apache.org/jira/browse/SPARK-7616. 6cabf3c [Yin Huai] Update unit test. 7e02910 [Yin Huai] Use metastore partition columns and do not hijack maybePartitionSpec. e9a03ec [Cheng Lian] Persists partition columns into metastore

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with apache#6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes apache#6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory

…tore Author: Yin Huai <yhuai@databricks.com> Author: Cheng Lian <lian@databricks.com> Closes apache#6285 from liancheng/spark-7763 and squashes the following commits: bb2829d [Yin Huai] Fix hashCode. d677f7d [Cheng Lian] Fixes Scala style issue 44b283f [Cheng Lian] Adds test case for SPARK-7616 6733276 [Yin Huai] Fix a bug that potentially causes https://issues.apache.org/jira/browse/SPARK-7616. 6cabf3c [Yin Huai] Update unit test. 7e02910 [Yin Huai] Use metastore partition columns and do not hijack maybePartitionSpec. e9a03ec [Cheng Lian] Persists partition columns into metastore

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with apache#6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes apache#6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory

…tore Author: Yin Huai <yhuai@databricks.com> Author: Cheng Lian <lian@databricks.com> Closes apache#6285 from liancheng/spark-7763 and squashes the following commits: bb2829d [Yin Huai] Fix hashCode. d677f7d [Cheng Lian] Fixes Scala style issue 44b283f [Cheng Lian] Adds test case for SPARK-7616 6733276 [Yin Huai] Fix a bug that potentially causes https://issues.apache.org/jira/browse/SPARK-7616. 6cabf3c [Yin Huai] Update unit test. 7e02910 [Yin Huai] Use metastore partition columns and do not hijack maybePartitionSpec. e9a03ec [Cheng Lian] Persists partition columns into metastore

liancheng mentioned this pull request May 20, 2015

[SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables #6287

Closed

Persists partition columns into metastore

e9a03ec

Use metastore partition columns and do not hijack maybePartitionSpec.

7e02910

yhuai force-pushed the spark-7763 branch from 2a38b5b to 7e02910 Compare May 20, 2015 23:15

yhuai added 2 commits May 20, 2015 21:46

Update unit test.

6cabf3c

Fix a bug that potentially causes https://issues.apache.org/jira/brow…

6733276

…se/SPARK-7616.

yhuai force-pushed the spark-7763 branch from 387153c to 6733276 Compare May 21, 2015 04:47

Fixes Scala style issue

d677f7d

liancheng changed the title ~~[SPARK-7763] [SQL] Persists partition columns into metastore~~ [SPARK-7763] [SPARK-7616] [SQL] Persists partition columns into metastore May 21, 2015

Fix hashCode.

bb2829d

asfgit closed this in 30f3f55 May 21, 2015

liancheng deleted the spark-7763 branch May 21, 2015 22:51

Comments

Conversation

liancheng commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!